1. Field of the Invention
The present invention relates generally to a system and method for enabling a blind or visually impaired user to use a graphical user interface, and more particularly, to a system and method for enabling a blind or visually impaired user to locate icons in a graphical user interface.
2. Description of the Prior Art
In recent years, there has been a move among computer application software developers toward graphical user interfaces (GUIS). In graphical user interfaces, objects are presented for users to manipulate in ways that are similar to the way that they are manipulated in the real work place. Objects, such as file cabinets, folders, documents, and printers, are displayed on the screen as icons. Users manipulate these objects with a mouse to perform desired operations. For example, to file a document in a folder that is located in a file cabinet in the real work place, the user opens the file cabinet, locates and opens the correct folder, and puts the document inside. In the electronic work place of the graphical user interface, the user performs a similar process. The user opens the file cabinet icon, locates the correct folder icon, and drops the document icon in the folder. Because this is an electronic environment, users do not have to open the folder to put the document in it. However, users have been able to use their knowledge of a real work place to perform this operation.
Normally sighted persons find graphical user interfaces intuitive and easy to work with. However, except for an occasional "beep" or "bong", graphical user interfaces are virtually silent and the vast majority of the information they provide to the user is visual. Thus, graphical user interfaces are essentially not usable by blind or severely visually impaired people.
Blind and visually impaired computer users now benefit from many forms of adaptive technology, including speech synthesis, large-print processing, braille desktop publishing, and voice recognition. However, presently, almost none of the foregoing tools is adapted for use with graphical user interfaces. It has been suggested that programmers could write software with built-in voice labels for icons. Lazzaro, Windows of Vulnerability, Byte Magazine, June, 1991 page 416. Various synthetic or recorded speech solutions for making computer display screen contents available to blind persons have been suggested, for example in Golding, et. al., IBM Technical Disclosure Bulletin, Vol. 26, No. 10B, pages 5633-5636 (March 1984), and Barnett, et. al., IBM Technical Disclosure Bulletin, Vol. 26, No. 10A, pages 4950-4951 (March 1984). Recently, there has been disclosed a prototype of a system called IBM Screen Reader/PM. Schwerdtfeger, Making the GUI Talk, Byte Magazine, December 1991, page 118. According to the Schwerdtfeger article, a user of the IBM Screen Reader/PM system can maneuver a mouse over the display and use the keyboard or a separate keypad, and a voice synthesizer will describe an icon the GUI has displayed or the graphical text shown on the screen. Additionally, there have been suggested systems that include a mouse with a braille transducer so that a blind user may read text and obtain certain tactile position feedback from the mouse. Comerford, IBM Technical Disclosure Bulletin Vol. 28, No. 3, page 1343 (August 1985), Affinito, et. al., IBM Technical Disclosure Bulletin Vol. 31, No. 12, page 386 (May 1989). However, while announcing various text items, either audibly or by means of a braille transducer in the mouse, may provide some information to blind user, it does not enable the user to navigate about and locate objects on the computer display screen.
There has been suggested an audible cursor positioning and pixel (picture element) status identification mechanism to help a user of an interactive computer graphics system locate data by using aural feedback to enhance visual feedback. As the cursor is stepped across the screen, an audible click is generated that varies in tone corresponding to the current status of each pixel encountered. With this combination in audible and visual cursor feedback, it becomes a simple task to identify the desired line by noting the change in tone as the cursor moves. For color display applications, each color is represented by a distinct tone so any single pixel may be distinguished from the surrounding pixels of a different color. Although the technique was originally developed for computer aided drafting, it has been suggested that this system is especially helpful for visually impaired or learning disabled users. Drumm, et. al., IBM Technical Disclosure Bulletin, Vol. 27, No. 48, page 2528 (September 1984). However, the foregoing disclosure does not suggest a means of enabling a blind user to navigate about or locate objects on the computer display screen.
Recently, in U.S. Pat. No. 5,223,828, issued Jun. 29, 1993, entitled "Method and System for Enabling a Blind Computer User to Handle Message Boxes in a Graphical User Interface", which is assigned to the assignee of the present application, a system has been proposed that permits a blind or visually impaired user to interact with message boxes within a graphical user interface. Each message box consists of an icon, explanatory text, and one or more "pushbuttons". The icon allows the user to identify visually the type of message. The text typically explains the situation and may provide assistance. The textual content may be a question or a statement. Pushbuttons provided within the message box typically allow the user to interact with the message box.
The system of U.S. Pat. No. 5,223,828 permits blind or visually impaired users to accommodate a message box by announcing the textual content of such a box when the message box first appears. Thereafter, the pushbuttons available to respond to the message box are announced in order from left to right. A homing signal was then provided for finding the message box. The homing signal increases in pitch as the mouse pointer approaches the message box. When the pointer enters the message box, the message box text and available pushbuttons are re-announced and the pointer is automatically moved to a default pushbutton. By using this system, a blind or visually impaired user may locate a message box within a computer system.
Another system and method is disclosed in U.S. patent application Ser. No. 08/022,788, filed Feb. 22, 1993, which is a continuation of abandoned U.S. patent application Ser. No. 07/746,840, filed Aug. 19, 1991, and entitled "Audio User Interface With Stereo and Filtered Sound Effects", which is assigned to the assignee of the present application. The system and method of application Ser. No. 07/746,840, permits a blind or visually impaired user to locate a mouse pointer or other graphical pointing device within the client area of a window within a graphical user interface by providing a stereo sound system and varying the intensity of the left and right audio channels to indicate the horizontal position of the mouse pointer. That system also proposes an increase in pitch of an associated sound to indicate the relative position of the pointer in the top/bottom access of the client area of the window.
Recently, in application Ser. No. 07/802,956, filed Dec. 5, 1991, entitled "Method and System for Enabling Blind or Visually Impaired Computer Users to Graphically Select Displayed Elements", which is assigned to the assignee of the present application, there is disclosed a method and system that may be used to enable a blind or visually impaired computer user to graphically select a displayed graphic element within a computer system display. A unique identifiable audible signal is associated with each displayed graphic element. A moveable cursor element or a pointer is displayed within the computer system display and a composite audible signal is periodically generated in response to the position of the moveable cursor element. The composite audible signal preferably includes elements of each identifiable audible signal associated with each displayed graphic element within a pre-determined radius of the location of the moveable cursor element. In one embodiment of that system and method, each displayed graphic element comprises multiple picture elements and the composite audible signal includes elements of each identifiable audible signal associated with each displayed graphic element having picture elements within a rotational sector of a circle having its origin at the moveable cursor element and a radius equal to the pre-determined radius.
In certain graphical user interfaces, frequently used objects that have system wide application are positioned in the background of the display screen, which is frequently called the "electronic desktop." Such objects may include printer icons and wastebin or shredder icons. The printer icons are provided so that a user may print a document or other objects by direct manipulation and shredder or wastebin icons are provided so that the user can delete a document or object by direct manipulation. Those icons are positioned on the desktop so that they will be readily available to the user without the user having to open a window that may contain them. Users are also typically given the ability to customize their interfaces by placing various objects of their own choosing on their desktop. Certain operating systems adapted for graphical user interfaces, such as IBM OS/2 Presentation Manager, provide a facility that automatically arranges the icons on the desktop in rows.
The prior art has provided certain tools by which a blind or visually impaired user may navigate within windows and find certain elements. However, there does not currently exist any tool by which a blind or visually impaired user can readily locate icons on the desktop. In the prior work on audio graphical user interfaces, the blind user has been able to wander about the screen in search of icons. When passing over an icon, certain sounds have been created or altered along with a verbal announcement of the identity of the icon using a text-to-speech synthesizer. However, this random searching process can be tiresome and time consuming to the user.